A theoretical basis for efficient computations with noisy spiking neurons

Network of neurons in the brain apply – unlike processors in our current generation of computer hardware – an event-based processing strategy, where short pulses (spikes) are emitted sparsely by neurons to signal the occurrence of an event at a particular point in time. Such spike-based computations promise to be substantially more power-efficient than traditional clocked processing schemes. However it turned out to be surprisingly difficult to design networks of spiking neurons that are able to carry out demanding computations. We present here a new theoretical framework for organizing computations of networks of spiking neurons. In particular, we show that a suitable design enables them to solve hard constraint satisfaction problems from the domains of planning/optimization and verification/logical inference. The underlying design principles employ noise as a computational resource. Nevertheless the timing of spikes (rather than just spike rates) plays an essential role in the resulting computations. Furthermore, one can demonstrate for the Traveling Salesman Problem a surprising computational advantage of networks of spiking neurons compared with traditional artificial neural networks and Gibbs sampling. The identification of such advantage has been a well-known open problem. The number of neurons in the brain lies in the same range as the number of transistor in a supercomputer. But whereas the brain consumes less then 30 Watt, a supercomputer consumes as much energy as a major part of a city. The power consumption has not only become a bottleneck for supercomputers, but for many applications and improvements of computing hardware, including the design of intelligent mobile devices. But how can one capture the drastically different style of computations by networks of neurons in the brain, and apply similar energy-efficient methods for the organization of computation in novel computing hardware? In particular, how can one carry out complex computations in massively parallel systems without a clock, that synchronizes the contributions of individual processors? When the membrane potential of a biological neuron crosses a threshold, the neuron emits a spike, i. e. a sudden voltage increase that lasts for 1 2 ms. Spikes occur asynchronously in continuous time and are communicated to numerous other neurons via synaptic connections with different strengths (“weights”). The effect of a spike from a pre-synaptic neuron l on a post-synaptic neuron k, the socalled post-synaptic potential (PSP), can be approximated as an additive contribution to its membrane potential. It is short-lived (10 20 ms) and can be either inhibitory or excitatory, depending on the sign of the synaptic weight wkl. It is a long-standing mystery how complex computations are organized in networks of spiking neurons in the brain, especially in view of ubiquitous sources of noise in neurons and synapses, and large trial-to-trial variability of network responses [1]. Hence it is not surprising, that attempts to port brain-inspired computational architectures into novel artificial computing hardware (see e.g. [2, 3, 4, 5, 6]) have had only limited success from the computational perspective. Whereas very nice results were achieved with artificial spike-based retinas [7] and cochleas [8], we are not aware of published methods for solving complex computational tasks efficiently by spike-based circuits. Powerful computations with spiking neurons have previously been demonstrated in [9]. But spiking neurons were Corresponding author: [email protected]. † These authors contributed equally to this work.